Running to You

Description

This animation shows the correlogram of a musical piece and its components. Separating the voice from such a mixture is the Holy Grail of the hearing effort at Apple. Human listeners can certainly understand the words that accompany this music; why can't machines? If we ever get to the point where we can separate out the voice from this background "noise" then a typical office environment should be easy.

This video clip shows three correlograms. The original song was recorded onto multi-track tape and then mixed so that the vocals were on one channel and the instrumentals were on the other channel. We digitized the two channels independently and added them together to make the sound track on this tape. Correlograms were then individually computed of the vocals, the instrumentals, and the mix.

The middle frame of this animation shows the correlogram of the sound you are actually listening to. The top frame shows the vocal track while the bottom frame shows the instrumental track. (Note that there is some bleeding of the sound from the instrumental track to the vocal track.)

It is unclear whether the correlogram is up to this task. We can certainly see evidence of the singer's pitch in the middle correlogram. This task is made harder because the pitch of the woman's voice harmonizes with the music. This means that her pitch will often land on top of the musical components and thus be hidden. More work remains to be done.

This piece, called "Running to You" was composed and performed by SSK (Steve Milne, Steve Squyres, and Kalena O'Malley). (c) 1985 by Steve Milne, Drew Wanderman. Used with permission.

Frequency
Time Delay

Transcript

[Singing Short Running To You]